Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Autocomplete] PPL Autocomplete #7810

Conversation

mengweieric
Copy link
Collaborator

@mengweieric mengweieric commented Aug 22, 2024

Description

  • PPL autocomplete support stage 1 to go into feature branch
    • Antlr parsing and suggesting for PPL query
    • Custom suggesting rules
    • Utilities for sharing among different languages

Issues Resolved

Screenshot

Testing the changes

Changelog

  • feat: Add OpenSearch PPL autocomplete to discover 2.0 with query enhancements

Check List

  • All tests pass
    • yarn test:jest
    • yarn test:jest_integration
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

Copy link
Contributor

❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

Copy link

codecov bot commented Aug 22, 2024

Codecov Report

Attention: Patch coverage is 29.41176% with 72 lines in your changes missing coverage. Please review.

Project coverage is 60.49%. Comparing base (23798a0) to head (61d5a5d).
Report is 1 commits behind head on feature/autocomplete-2.0.

Files Patch % Lines
...ata/public/antlr/opensearch_ppl/code_completion.ts 12.00% 22 Missing ⚠️
src/plugins/data/public/antlr/shared/utils.ts 4.54% 21 Missing ⚠️
...ntlr/opensearch_ppl/opensearch_ppl_autocomplete.ts 44.11% 19 Missing ⚠️
...ata/public/antlr/shared/general_error_listerner.ts 0.00% 6 Missing ⚠️
src/plugins/data/public/plugin.ts 0.00% 4 Missing ⚠️
Additional details and impacted files
@@                     Coverage Diff                      @@
##           feature/autocomplete-2.0    #7810      +/-   ##
============================================================
- Coverage                     63.83%   60.49%   -3.35%     
============================================================
  Files                          3661     3657       -4     
  Lines                         81345    86844    +5499     
  Branches                      12978    13324     +346     
============================================================
+ Hits                          51926    52532     +606     
- Misses                        26234    31143    +4909     
+ Partials                       3185     3169      -16     
Flag Coverage Δ
Linux_1 28.65% <29.41%> (-1.50%) ⬇️
Linux_2 55.89% <ø> (+0.02%) ⬆️
Linux_3 37.66% <29.41%> (-2.74%) ⬇️
Linux_4 29.49% <29.41%> (-1.86%) ⬇️
Windows_1 28.66% <29.41%> (-1.50%) ⬇️
Windows_2 55.85% <ø> (+0.02%) ⬆️
Windows_3 37.66% <29.41%> (-2.74%) ⬇️
Windows_4 29.49% <29.41%> (-1.86%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Eric <[email protected]>
Signed-off-by: Eric <[email protected]>
@mengweieric mengweieric changed the title [Draft] PPL Autocomplete [Autocomplete] PPL Autocomplete Aug 22, 2024
@@ -85,7 +85,8 @@
"cypress:run-with-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=true,openSearchUrl=https://localhost:9200,WAIT_FOR_LOADER_BUFFER_MS=500",
"osd:ciGroup10": "echo \"dashboard_sanity_test_spec.js\"",
"osd:ciGroup11": "echo \"apps/vis_builder/*.js\"",
"generate:opensearchsqlantlr": "./node_modules/antlr4ng-cli/index.js -Dlanguage=TypeScript -o ./src/plugins/data/public/antlr/opensearch_sql/.generated -visitor -no-listener -Xexact-output-dir ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLLexer.g4 ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLParser.g4"
"generate:opensearchsqlantlr": "./node_modules/antlr4ng-cli/index.js -Dlanguage=TypeScript -o ./src/plugins/data/public/antlr/opensearch_sql/.generated -visitor -no-listener -Xexact-output-dir ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLLexer.g4 ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLParser.g4",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice. i see what you did. another backlog item could be moving the scripts to a file to the scripts folder and then call the script like yarn generate:antler --sql

import { QuerySuggestion } from '../../autocomplete';
import { IndexPattern, IndexPatternField } from '../../index_patterns';

const fetchFieldSuggestions = (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think there can be a space in the language manager we where we register code completion stuff so that these files can be put in the query enhancements plugin.

cc: @ashwin-pc, @abbyhu2000 what do you think?

@sejli
Copy link
Member

sejli commented Aug 26, 2024

Merging into feature branch.

@sejli sejli merged commit 3ae20ef into opensearch-project:feature/autocomplete-2.0 Aug 26, 2024
56 of 66 checks passed
paulstn pushed a commit to paulstn/OpenSearch-Dashboards that referenced this pull request Aug 27, 2024
* add initial ppl autocomplete

Signed-off-by: Eric <[email protected]>

* untrack .antlr files

Signed-off-by: Eric <[email protected]>

* ignore intermediate grammar auto-gen files

Signed-off-by: Eric <[email protected]>

* add rules and related functionalities

Signed-off-by: Eric <[email protected]>

* Changeset file for PR opensearch-project#7810 created/updated

* Changeset file for PR opensearch-project#7810 created/updated

* minor comment cleanning

Signed-off-by: Eric <[email protected]>

* add ppl generation command

Signed-off-by: Eric <[email protected]>

* add rules

Signed-off-by: Eric <[email protected]>

* correct typo

Signed-off-by: Eric <[email protected]>

* fix inserting text issue

Signed-off-by: Eric <[email protected]>

* remove colon for PPL field

Signed-off-by: Eric <[email protected]>

---------

Signed-off-by: Eric <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
paulstn pushed a commit to paulstn/OpenSearch-Dashboards that referenced this pull request Aug 30, 2024
* add initial ppl autocomplete

Signed-off-by: Eric <[email protected]>

* untrack .antlr files

Signed-off-by: Eric <[email protected]>

* ignore intermediate grammar auto-gen files

Signed-off-by: Eric <[email protected]>

* add rules and related functionalities

Signed-off-by: Eric <[email protected]>

* Changeset file for PR opensearch-project#7810 created/updated

* Changeset file for PR opensearch-project#7810 created/updated

* minor comment cleanning

Signed-off-by: Eric <[email protected]>

* add ppl generation command

Signed-off-by: Eric <[email protected]>

* add rules

Signed-off-by: Eric <[email protected]>

* correct typo

Signed-off-by: Eric <[email protected]>

* fix inserting text issue

Signed-off-by: Eric <[email protected]>

* remove colon for PPL field

Signed-off-by: Eric <[email protected]>

---------

Signed-off-by: Eric <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
paulstn pushed a commit to paulstn/OpenSearch-Dashboards that referenced this pull request Aug 30, 2024
* add initial ppl autocomplete

Signed-off-by: Eric <[email protected]>

* untrack .antlr files

Signed-off-by: Eric <[email protected]>

* ignore intermediate grammar auto-gen files

Signed-off-by: Eric <[email protected]>

* add rules and related functionalities

Signed-off-by: Eric <[email protected]>

* Changeset file for PR opensearch-project#7810 created/updated

* Changeset file for PR opensearch-project#7810 created/updated

* minor comment cleanning

Signed-off-by: Eric <[email protected]>

* add ppl generation command

Signed-off-by: Eric <[email protected]>

* add rules

Signed-off-by: Eric <[email protected]>

* correct typo

Signed-off-by: Eric <[email protected]>

* fix inserting text issue

Signed-off-by: Eric <[email protected]>

* remove colon for PPL field

Signed-off-by: Eric <[email protected]>

---------

Signed-off-by: Eric <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
ashwin-pc pushed a commit that referenced this pull request Aug 30, 2024
* [Autocomplete - SQL] Minor interface change to add suggestion type and move suggestion provider registration location (#7758)

* add table/source as prefix to suggested fields

Signed-off-by: Eric <[email protected]>

* add type to column

Signed-off-by: Eric <[email protected]>

* move registeration to osd/monaco

Signed-off-by: Eric <[email protected]>

* add detail

Signed-off-by: Eric <[email protected]>

* Changeset file for PR #7758 created/updated

---------

Signed-off-by: Eric <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>

* [Autocomplete] PPL Autocomplete (#7810)

* add initial ppl autocomplete

Signed-off-by: Eric <[email protected]>

* untrack .antlr files

Signed-off-by: Eric <[email protected]>

* ignore intermediate grammar auto-gen files

Signed-off-by: Eric <[email protected]>

* add rules and related functionalities

Signed-off-by: Eric <[email protected]>

* Changeset file for PR #7810 created/updated

* Changeset file for PR #7810 created/updated

* minor comment cleanning

Signed-off-by: Eric <[email protected]>

* add ppl generation command

Signed-off-by: Eric <[email protected]>

* add rules

Signed-off-by: Eric <[email protected]>

* correct typo

Signed-off-by: Eric <[email protected]>

* fix inserting text issue

Signed-off-by: Eric <[email protected]>

* remove colon for PPL field

Signed-off-by: Eric <[email protected]>

---------

Signed-off-by: Eric <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>

* small interface updates

Signed-off-by: Paul Sebastian <[email protected]>

* small type update

Signed-off-by: Paul Sebastian <[email protected]>

* make inclusion for pipe, comma, and equal tokens

Signed-off-by: Paul Sebastian <[email protected]>

* refactor and generalize field fetcher for dql and ppl, including details

Signed-off-by: Paul Sebastian <[email protected]>

* use field fetching util for sql and update sugg detail for dql

Signed-off-by: Paul Sebastian <[email protected]>

* detail for ppl

Signed-off-by: Paul Sebastian <[email protected]>

* create range parameter to help identify suggestions with whitespace, implemented dql value WS suggs

Signed-off-by: Paul Sebastian <[email protected]>

* single line editor overflow initial override for sugg window

Signed-off-by: Paul Sebastian <[email protected]>

* update dql tests to account for details and value ranges

Signed-off-by: Paul Sebastian <[email protected]>

---------

Signed-off-by: Eric <[email protected]>
Signed-off-by: Paul Sebastian <[email protected]>
Co-authored-by: Eric Wei <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 30, 2024
* [Autocomplete - SQL] Minor interface change to add suggestion type and move suggestion provider registration location (#7758)

* add table/source as prefix to suggested fields

Signed-off-by: Eric <[email protected]>

* add type to column

Signed-off-by: Eric <[email protected]>

* move registeration to osd/monaco

Signed-off-by: Eric <[email protected]>

* add detail

Signed-off-by: Eric <[email protected]>

* Changeset file for PR #7758 created/updated

---------

Signed-off-by: Eric <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>

* [Autocomplete] PPL Autocomplete (#7810)

* add initial ppl autocomplete

Signed-off-by: Eric <[email protected]>

* untrack .antlr files

Signed-off-by: Eric <[email protected]>

* ignore intermediate grammar auto-gen files

Signed-off-by: Eric <[email protected]>

* add rules and related functionalities

Signed-off-by: Eric <[email protected]>

* Changeset file for PR #7810 created/updated

* Changeset file for PR #7810 created/updated

* minor comment cleanning

Signed-off-by: Eric <[email protected]>

* add ppl generation command

Signed-off-by: Eric <[email protected]>

* add rules

Signed-off-by: Eric <[email protected]>

* correct typo

Signed-off-by: Eric <[email protected]>

* fix inserting text issue

Signed-off-by: Eric <[email protected]>

* remove colon for PPL field

Signed-off-by: Eric <[email protected]>

---------

Signed-off-by: Eric <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>

* small interface updates

Signed-off-by: Paul Sebastian <[email protected]>

* small type update

Signed-off-by: Paul Sebastian <[email protected]>

* make inclusion for pipe, comma, and equal tokens

Signed-off-by: Paul Sebastian <[email protected]>

* refactor and generalize field fetcher for dql and ppl, including details

Signed-off-by: Paul Sebastian <[email protected]>

* use field fetching util for sql and update sugg detail for dql

Signed-off-by: Paul Sebastian <[email protected]>

* detail for ppl

Signed-off-by: Paul Sebastian <[email protected]>

* create range parameter to help identify suggestions with whitespace, implemented dql value WS suggs

Signed-off-by: Paul Sebastian <[email protected]>

* single line editor overflow initial override for sugg window

Signed-off-by: Paul Sebastian <[email protected]>

* update dql tests to account for details and value ranges

Signed-off-by: Paul Sebastian <[email protected]>

---------

Signed-off-by: Eric <[email protected]>
Signed-off-by: Paul Sebastian <[email protected]>
Co-authored-by: Eric Wei <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
(cherry picked from commit 0245540)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
ashwin-pc pushed a commit that referenced this pull request Aug 30, 2024
* [Autocomplete - SQL] Minor interface change to add suggestion type and move suggestion provider registration location (#7758)

* add table/source as prefix to suggested fields



* add type to column



* move registeration to osd/monaco



* add detail



* Changeset file for PR #7758 created/updated

---------




* [Autocomplete] PPL Autocomplete (#7810)

* add initial ppl autocomplete



* untrack .antlr files



* ignore intermediate grammar auto-gen files



* add rules and related functionalities



* Changeset file for PR #7810 created/updated

* Changeset file for PR #7810 created/updated

* minor comment cleanning



* add ppl generation command



* add rules



* correct typo



* fix inserting text issue



* remove colon for PPL field



---------




* small interface updates



* small type update



* make inclusion for pipe, comma, and equal tokens



* refactor and generalize field fetcher for dql and ppl, including details



* use field fetching util for sql and update sugg detail for dql



* detail for ppl



* create range parameter to help identify suggestions with whitespace, implemented dql value WS suggs



* single line editor overflow initial override for sugg window



* update dql tests to account for details and value ranges



---------





(cherry picked from commit 0245540)

Signed-off-by: Eric <[email protected]>
Signed-off-by: Paul Sebastian <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric Wei <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants